Papers with monotonicity loss function
On Biasing Transformer Attention Towards Monotonicity (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work has focused on learning monotonic attention behavior via specialized attention functions or pretraining. |
| Approach: | They introduce a monotonicity loss function compatible with standard attention mechanisms and test it on sequence-to-sequence tasks. |
| Outcome: | The proposed monotonicity loss function can achieve largely monotonic behavior on grapheme-to-phoneme conversion, morphological inflection, transliteration, and dialect normalization tasks. |